Multimodal sensor fusion for real-time location-dependent defect detection in laser-directed energy deposition

Real-time defect detection is crucial in laser-directed energy deposition (L-DED) additive manufacturing (AM). Traditional in-situ monitoring approach utilizes a single sensor (i.e., acoustic, visual, or thermal sensor) to capture the complex process dynamic behaviors, which is insufficient for defect detection with high accuracy and robustness. This paper proposes a novel multimodal sensor fusion method for real-time location-dependent defect detection in the robotic L-DED process. The multimodal fusion sources include a microphone sensor capturing the laser-material interaction sound and a visible spectrum CCD camera capturing the coaxial melt pool images. A hybrid convolutional neural network (CNN) is proposed to fuse acoustic and visual data. The key novelty in this study is that the traditional manual feature extraction procedures are no longer required, and the raw melt pool images and acoustic signals are fused directly by the hybrid CNN model, which achieved the highest defect prediction accuracy (98.5 %) without the thermal sensing modality. Moreover, unlike previous region-based quality prediction, the proposed hybrid CNN can detect the onset of defect occurrences. The defect prediction outcomes are synchronized and registered with in-situ acquired robot tool-center-point (TCP) data, which enables localized defect identification. The proposed multimodal sensor fusion method offers a robust solution for in-situ defect detection.Comment: 8 pages, 10 figures. This paper has been accepted to be published in the proceedings of IDETC-CIE 202

Symphony: Optimized Model Serving using Centralized Orchestration

The orchestration of deep neural network (DNN) model inference on GPU clusters presents two significant challenges: achieving high accelerator efficiency given the batching properties of model inference while meeting latency service level objectives (SLOs), and adapting to workload changes both in terms of short-term fluctuations and long-term resource allocation. To address these challenges, we propose Symphony, a centralized scheduling system that can scale to millions of requests per second and coordinate tens of thousands of GPUs. Our system utilizes a non-work-conserving scheduling algorithm capable of achieving high batch efficiency while also enabling robust autoscaling. Additionally, we developed an epoch-scale algorithm that allocates models to sub-clusters based on the compute and memory needs of the models. Through extensive experiments, we demonstrate that Symphony outperforms prior systems by up to 4.7x higher goodput

Punica: Multi-Tenant LoRA Serving

Low-rank adaptation (LoRA) has become an important and popular method to adapt pre-trained models to specific domains. We present Punica, a system to serve multiple LoRA models in a shared GPU cluster. Punica contains a new CUDA kernel design that allows batching of GPU operations for different LoRA models. This allows a GPU to hold only a single copy of the underlying pre-trained model when serving multiple, different LoRA models, significantly enhancing GPU efficiency in terms of both memory and computation. Our scheduler consolidates multi-tenant LoRA serving workloads in a shared GPU cluster. With a fixed-sized GPU cluster, our evaluations show that Punica achieves 12x higher throughput in serving multiple LoRA models compared to state-of-the-art LLM serving systems while only adding 2ms latency per token. Punica is open source at https://github.com/punica-ai/punica

In-situ crack and keyhole pore detection in laser directed energy deposition through acoustic signal and deep learning

Cracks and keyhole pores are detrimental defects in alloys produced by laser directed energy deposition (LDED). Laser-material interaction sound may hold information about underlying complex physical events such as crack propagation and pores formation. However, due to the noisy environment and intricate signal content, acoustic-based monitoring in LDED has received little attention. This paper proposes a novel acoustic-based in-situ defect detection strategy in LDED. The key contribution of this study is to develop an in-situ acoustic signal denoising, feature extraction, and sound classification pipeline that incorporates convolutional neural networks (CNN) for online defect prediction. Microscope images are used to identify locations of the cracks and keyhole pores within a part. The defect locations are spatiotemporally registered with acoustic signal. Various acoustic features corresponding to defect-free regions, cracks, and keyhole pores are extracted and analysed in time-domain, frequency-domain, and time-frequency representations. The CNN model is trained to predict defect occurrences using the Mel-Frequency Cepstral Coefficients (MFCCs) of the lasermaterial interaction sound. The CNN model is compared to various classic machine learning models trained on the denoised acoustic dataset and raw acoustic dataset. The validation results shows that the CNN model trained on the denoised dataset outperforms others with the highest overall accuracy (89%), keyhole pore prediction accuracy (93%), and AUC-ROC score (98%). Furthermore, the trained CNN model can be deployed into an in-house developed software platform for online quality monitoring. The proposed strategy is the first study to use acoustic signals with deep learning for insitu defect detection in LDED process.Comment: 36 Pages, 16 Figures, accepted at journal Additive Manufacturin

Cloning, expression and characterization of alcohol dehydrogenases in the silkworm Bombyx mori

Alcohol dehydrogenases (ADH) are a class of enzymes that catalyze the reversible oxidation of alcohols to corresponding aldehydes or ketones, by using either nicotinamide adenine dinucleotide (NAD) or nicotinamide adenine dinucleotide phosphate (NADP), as coenzymes. In this study, a short-chain ADH gene was identified in Bombyx mori by 5′-RACE PCR. This is the first time the coding region of BmADH has been cloned, expressed, purified and then characterized. The cDNA fragment encoding the BmADH protein was amplified from a pool of silkworm cDNAs by PCR, and then cloned into E. coli expression vector pET-30a(+). The recombinant His-tagged BmADH protein was expressed in E. coli BL21 (DE3), and then purified by metal chelating affinity chromatography. The soluble recombinant BmADH, produced at low-growth temperature, was instrumental in catalyzing the ethanol-dependent reduction of NAD+, thereby indicating ethanol as one of the substrates of BmADH

Process monitoring and control for laser-aided additive manufacturing

In-situ monitoring and closed-loop control are two critical methodologies for quality assurance in laser-aided additive manufacturing (LAAM). In particular, geometric conditions of fabricated parts need to be monitored in real-time so that surface defects can be detected and corrected early to avoid further deterioration. Closed-loop control of laser power based on melt pool feedback can enhance the mechanical integrity and reduce defect occurrences. However, the state-of-the-art surface quality inspection techniques require data post-processing with undesirable process intermittence, and conventional closed-loop control requires pre-build parameter optimization, which is cumbersome and time-consuming. In this research, an AI-assisted rapid surface defect detection method with in-situ point cloud data processing and semi-supervised machine learning is proposed, and a novel data-driven adaptive controller with automatic parameter tuning algorithm is presented. The surface monitoring and adaptive control algorithms are implemented in a Robot Operating System (ROS) based software platform. The integrated multi-nodal software architecture enables on-the-fly surface defect detection and real-time laser power control without process intermittence. A defect correction algorithm is implemented to automatically generate repairing tool path if geometric distortions were identified during the deposition process. The main advantage of the proposed monitoring and control system is its efficiency and adaptivity for industrial adoption. Multiple subprocesses can run simultaneously, and surface defects are detected and corrected without human intervention. The proposed control technique is robust to various deposition conditions. Pre-build parameter optimization is not needed even when the deposition material or the parts’ geometries are changed. Experimental results have shown 93.15% defect identification accuracy based on the proposed semi-supervised model and significant improvement in dimensional accuracy of the printed parts attributed to the proposed data-driven adaptive control method.Bachelor of Engineering (Mechanical Engineering